{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example Seldon Core Deployments using Ksonnet\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequistes\n",
    "You will need\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - A running Kubernetes cluster with kubectl authenticated\n",
    " - [python grpc tools](https://grpc.io/docs/quickstart/python.html)\n",
    " - [ksonnet client](https://ksonnet.io/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Kubernetes Cluster\n",
    "\n",
    "Follow the [Kubernetes documentation to create a cluster](https://kubernetes.io/docs/setup/).\n",
    "\n",
    "Once created ensure ```kubectl``` is authenticated against the running cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** If running on GCP then run following to add cluster-admin to your user account **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clusterrolebinding.rbac.authorization.k8s.io/my-cluster-admin-binding created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create clusterrolebinding my-cluster-admin-binding --clusterrole=cluster-admin --user=$(gcloud info --format=\"value(config.account)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On most clusters you will probably need to add cluster-admin privledges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"minikube\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Seldon Core\n",
    "Create a ksonnet app and install the prototypes from our registry."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Using context \"minikube\" from kubeconfig file \"/home/clive/.kube/config\" \n",
      "\u001b[36mINFO\u001b[0m Creating environment \"default\" with namespace \"seldon\", pointing to \"version:v1.10.0\" cluster at address \"https://192.168.39.120:8443\" \n",
      "\u001b[36mINFO\u001b[0m Generating ksonnet-lib data at path '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/lib/ksonnet-lib/v1.10.0' \n"
     ]
    }
   ],
   "source": [
    "!rm -rf my-ml-deployment && ks init my-ml-deployment "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Retrieved 16 files                           \n",
      "\u001b[36mINFO\u001b[0m Writing component at '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/seldon-core.jsonnet' \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks registry add seldon-core ../../seldon-core && \\\n",
    "    ks pkg install seldon-core/seldon-core@master && \\\n",
    "    ks generate seldon-core seldon-core --withApife=false --withAmbassador=true --namespace=seldon --withRbac=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Applying serviceaccounts seldon.seldon       \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent serviceaccounts seldon.seldon \n",
      "\u001b[36mINFO\u001b[0m Applying clusterroles seldon-crd             \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent clusterroles seldon-crd \n",
      "\u001b[36mINFO\u001b[0m Applying roles seldon.seldon-local           \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent roles seldon.seldon-local \n",
      "\u001b[36mINFO\u001b[0m Applying rolebindings seldon.seldon          \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent rolebindings seldon.seldon \n",
      "\u001b[36mINFO\u001b[0m Applying clusterrolebindings seldon.seldon   \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent clusterrolebindings seldon.seldon \n",
      "\u001b[36mINFO\u001b[0m Applying deployments seldon.seldon-core-seldon-cluster-manager \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent deployments seldon.seldon-core-seldon-cluster-manager \n",
      "\u001b[36mINFO\u001b[0m Applying deployments seldon.seldon-core-redis \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent deployments seldon.seldon-core-redis \n",
      "\u001b[36mINFO\u001b[0m Applying services seldon.seldon-core-redis   \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent services seldon.seldon-core-redis \n",
      "\u001b[36mINFO\u001b[0m Applying customresourcedefinitions seldondeployments.machinelearning.seldon.io \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent customresourcedefinitions seldondeployments.machinelearning.seldon.io \n",
      "\u001b[36mINFO\u001b[0m Applying roles seldon.ambassador             \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent roles seldon.ambassador \n",
      "\u001b[36mINFO\u001b[0m Applying rolebindings seldon.ambassador      \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent rolebindings seldon.ambassador \n",
      "\u001b[36mINFO\u001b[0m Applying deployments seldon.seldon-core-ambassador \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent deployments seldon.seldon-core-ambassador \n",
      "\u001b[36mINFO\u001b[0m Applying services seldon.seldon-core-ambassador \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent services seldon.seldon-core-ambassador \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "      ks apply default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"seldon-core-seldon-cluster-manager\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-seldon-cluster-manager\" successfully rolled out\n",
      "Waiting for deployment \"seldon-core-ambassador\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-ambassador\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-core-seldon-cluster-manager\n",
    "!kubectl rollout status deploy/seldon-core-ambassador"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up REST and gRPC methods\n",
    "\n",
    "**Ensure you port forward ambassador**:\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l service=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install gRPC modules for the prediction protos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cp ../proto/prediction.proto ./proto\n",
    "!python -m grpc.tools.protoc -I. --python_out=. --grpc_python_out=. ./proto/prediction.proto"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Single Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Writing component at '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/mymodel.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Applying seldondeployments seldon.mymodel    \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent seldondeployments seldon.mymodel \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks generate seldon-serve-simple-v1alpha2 mymodel --image seldonio/mock_classifier:1.0 && \\\n",
    "    ks apply default -c mymodel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---\r\n",
      "apiVersion: machinelearning.seldon.io/v1alpha2\r\n",
      "kind: SeldonDeployment\r\n",
      "metadata:\r\n",
      "  labels:\r\n",
      "    app: seldon\r\n",
      "    ksonnet.io/component: mymodel\r\n",
      "  name: mymodel\r\n",
      "  namespace: seldon\r\n",
      "spec:\r\n",
      "  annotations:\r\n",
      "    deployment_version: v1\r\n",
      "    project_name: mymodel\r\n",
      "  name: mymodel\r\n",
      "  predictors:\r\n",
      "  - annotations:\r\n",
      "      predictor_version: v1\r\n",
      "    componentSpecs:\r\n",
      "    - spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/mock_classifier:1.0\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: mymodel\r\n",
      "          volumeMounts: []\r\n",
      "        imagePullSecrets: []\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "        volumes: []\r\n",
      "    graph:\r\n",
      "      children: []\r\n",
      "      endpoint:\r\n",
      "        type: REST\r\n",
      "      name: mymodel\r\n",
      "      type: MODEL\r\n",
      "    name: mymodel\r\n",
      "    replicas: 1\r\n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks show default -c mymodel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mymodel-mymodel-mymodel-0\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-mymodel-0\" successfully rolled out\n",
      "Waiting for deployment \"mymodel-mymodel-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymodel-mymodel-svc-orch\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymodel-mymodel-mymodel-0\n",
    "!kubectl rollout status deploy/mymodel-mymodel-svc-orch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check status of deployment before continuing. **ReplicasAvailable must be equal to 1**  First time might take some time to download images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[state:Available predictorStatus:[map[name:mymodel-mymodel-mymodel-0 replicas:1 replicasAvailable:1] map[name:mymodel-mymodel-svc-orch replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments mymodel -o jsonpath='{.status}' -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"e9rps1fbsrn4ipb047t3onbq9o\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07284568533256051]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymodel\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "meta {\n",
      "  puid: \"h0r8bam9qmeqpg84lq65d7mi6u\"\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 1\n",
      "    values: 0.0806679270404733\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymodel\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding Authentication\n",
    "We will add the example authentication from the Ambassador tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "service/example-auth created\r\n",
      "deployment.extensions/example-auth created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/ambassador-auth-service-setup.yaml -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Need to wait until running before adding Ambassador config **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME                                                  READY   STATUS    RESTARTS   AGE\r\n",
      "example-auth-776ddb4c48-tvm2b                         1/1     Running   0          5s\r\n",
      "mymodel-mymodel-mymodel-0-76f8fd9746-fzfs8            1/1     Running   0          12m\r\n",
      "mymodel-mymodel-svc-orch-7dd77ffbcf-77wxm             1/1     Running   0          12m\r\n",
      "seldon-core-ambassador-778c58bf5d-vqqb7               2/2     Running   0          1h\r\n",
      "seldon-core-redis-5c498fc545-69pv8                    1/1     Running   0          1h\r\n",
      "seldon-core-seldon-cluster-manager-55cdc6f6bb-c9fqn   1/1     Running   0          1h\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl get pods -n seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "service/example-auth configured\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/ambassador-auth-service-config.yaml -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show failed request when auth is running"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [401]>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rest_request_ambassador(\"mymodel\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show successful request with auth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"ebfv93g1n2c777q3v3lp2bi3k4\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07878008242254023]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador_auth(\"mymodel\",\"username\",\"password\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "service \"example-auth\" deleted\r\n",
      "deployment.extensions \"example-auth\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/ambassador-auth-service-setup.yaml -n seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Deleting seldondeployments seldon.mymodel    \n",
      "\u001b[36mINFO\u001b[0m removing environment component                \u001b[36mcomponent-name\u001b[0m=mymodel\n",
      "\u001b[36mINFO\u001b[0m Removing component parameter references ...  \n",
      "\u001b[36mINFO\u001b[0m Deleting component 'mymodel' at path '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/mymodel.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Successfully deleted component 'mymodel'     \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks delete default -c mymodel && \\\n",
    "    ks component rm mymodel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve AB Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Writing component at '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/myabtest.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Applying seldondeployments seldon.myabtest   \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent seldondeployments seldon.myabtest \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks generate seldon-abtest-v1alpha2 myabtest --imageA seldonio/mock_classifier:1.0 --imageB seldonio/mock_classifier:1.0 && \\\n",
    "    ks apply default -c myabtest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---\r\n",
      "apiVersion: machinelearning.seldon.io/v1alpha2\r\n",
      "kind: SeldonDeployment\r\n",
      "metadata:\r\n",
      "  labels:\r\n",
      "    app: seldon\r\n",
      "    ksonnet.io/component: myabtest\r\n",
      "  name: myabtest\r\n",
      "  namespace: seldon\r\n",
      "spec:\r\n",
      "  annotations:\r\n",
      "    deployment_version: v1\r\n",
      "    project_name: myabtest\r\n",
      "  name: myabtest\r\n",
      "  predictors:\r\n",
      "  - componentSpecs:\r\n",
      "    - metadata:\r\n",
      "        labels:\r\n",
      "          version: v2\r\n",
      "      spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/mock_classifier:1.0\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: classifier-1\r\n",
      "          volumeMounts: []\r\n",
      "        imagePullSecrets: []\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "        volumes: []\r\n",
      "    - metadata:\r\n",
      "        labels:\r\n",
      "          version: v2\r\n",
      "      spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/mock_classifier:1.0\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: classifier-2\r\n",
      "          volumeMounts: []\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "        volumes: []\r\n",
      "    graph:\r\n",
      "      children:\r\n",
      "      - children: []\r\n",
      "        endpoint:\r\n",
      "          type: REST\r\n",
      "        name: classifier-1\r\n",
      "        type: MODEL\r\n",
      "      - children: []\r\n",
      "        endpoint:\r\n",
      "          type: REST\r\n",
      "        name: classifier-2\r\n",
      "        type: MODEL\r\n",
      "      endpoint: {}\r\n",
      "      implementation: RANDOM_ABTEST\r\n",
      "      name: random-ab-test\r\n",
      "      parameters:\r\n",
      "      - name: ratioA\r\n",
      "        type: FLOAT\r\n",
      "        value: \"0.5\"\r\n",
      "    name: myabtest\r\n",
      "    replicas: 1\r\n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks show default -c myabtest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"myabtest-myabtest-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"myabtest-myabtest-svc-orch\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/myabtest-myabtest-svc-orch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[state:Available predictorStatus:[map[name:myabtest-myabtest-svc-orch replicas:1 replicasAvailable:1] map[name:myabtest-myabtest-classifier-1-0 replicas:1 replicasAvailable:1] map[name:myabtest-myabtest-classifier-2-1 replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments myabtest -o jsonpath='{.status}' -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"u4r8h10rs8fcc1lifvepol5p31\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"random-ab-test\": 0\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.06803419031164869]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"myabtest\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "meta {\n",
      "  puid: \"phtcccgvriotfvajsj6k6gq3md\"\n",
      "  routing {\n",
      "    key: \"random-ab-test\"\n",
      "    value: 1\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 1\n",
      "    values: 0.08371109781590454\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_ambassador(\"myabtest\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Deleting seldondeployments seldon.myabtest   \n",
      "\u001b[36mINFO\u001b[0m removing environment component                \u001b[36mcomponent-name\u001b[0m=myabtest\n",
      "\u001b[36mINFO\u001b[0m Removing component parameter references ...  \n",
      "\u001b[36mINFO\u001b[0m Deleting component 'myabtest' at path '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/myabtest.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Successfully deleted component 'myabtest'    \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks delete default -c myabtest && \\\n",
    "    ks component rm myabtest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Multi-Armed Bandit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Writing component at '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/mymab.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Applying seldondeployments seldon.mymab      \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent seldondeployments seldon.mymab \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks generate seldon-mab-v1alpha2 mymab --imageA seldonio/mock_classifier:1.0 --imageB seldonio/mock_classifier:1.0 && \\\n",
    "    ks apply default -c mymab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---\r\n",
      "apiVersion: machinelearning.seldon.io/v1alpha2\r\n",
      "kind: SeldonDeployment\r\n",
      "metadata:\r\n",
      "  labels:\r\n",
      "    app: seldon\r\n",
      "    ksonnet.io/component: mymab\r\n",
      "  name: mymab\r\n",
      "  namespace: seldon\r\n",
      "spec:\r\n",
      "  annotations:\r\n",
      "    deployment_version: v1\r\n",
      "    project_name: mymab\r\n",
      "  name: mymab\r\n",
      "  predictors:\r\n",
      "  - componentSpecs:\r\n",
      "    - metadata:\r\n",
      "        labels:\r\n",
      "          version: v1\r\n",
      "      spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/mock_classifier:1.0\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: classifier-1\r\n",
      "          volumeMounts: []\r\n",
      "        imagePullSecrets: []\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "        volumes: []\r\n",
      "    - metadata:\r\n",
      "        labels:\r\n",
      "          version: v2\r\n",
      "      spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/mock_classifier:1.0\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: classifier-2\r\n",
      "          volumeMounts: []\r\n",
      "        imagePullSecrets: []\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "        volumes: []\r\n",
      "    - spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/mab_epsilon_greedy:1.1\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: eg-router\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "    graph:\r\n",
      "      children:\r\n",
      "      - children: []\r\n",
      "        endpoint:\r\n",
      "          type: REST\r\n",
      "        name: classifier-1\r\n",
      "        type: MODEL\r\n",
      "      - children: []\r\n",
      "        endpoint:\r\n",
      "          type: REST\r\n",
      "        name: classifier-2\r\n",
      "        type: MODEL\r\n",
      "      name: eg-router\r\n",
      "      parameters:\r\n",
      "      - name: n_branches\r\n",
      "        type: INT\r\n",
      "        value: \"2\"\r\n",
      "      - name: epsilon\r\n",
      "        type: FLOAT\r\n",
      "        value: \"0.2\"\r\n",
      "      - name: verbose\r\n",
      "        type: BOOL\r\n",
      "        value: \"1\"\r\n",
      "      type: ROUTER\r\n",
      "    name: mymab\r\n",
      "    replicas: 1\r\n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks show default -c mymab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mymab-mymab-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymab-mymab-svc-orch\" successfully rolled out\n",
      "Waiting for deployment \"mymab-mymab-eg-router-2\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mymab-mymab-eg-router-2\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mymab-mymab-svc-orch\n",
    "!kubectl rollout status deploy/mymab-mymab-eg-router-2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:mymab-mymab-svc-orch replicas:1 replicasAvailable:1] map[name:mymab-mymab-classifier-1-0 replicas:1 replicasAvailable:1] map[name:mymab-mymab-classifier-2-1 replicas:1 replicasAvailable:1] map[replicasAvailable:0 name:mymab-mymab-eg-router-2 replicas:1]] state:Creating]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments mymab -o jsonpath='{.status}' -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"48ukr3h36i823nt1sr3ak6dr5u\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"eg-router\": 0\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07239229629453742]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"mymab\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"e8jqq7btlq9qng4cdia90lj5c5\"\n",
       "  routing {\n",
       "    key: \"eg-router\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.07295662551802977\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"mymab\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Deleting seldondeployments seldon.mymab      \n",
      "\u001b[36mINFO\u001b[0m removing environment component                \u001b[36mcomponent-name\u001b[0m=mymab\n",
      "\u001b[36mINFO\u001b[0m Removing component parameter references ...  \n",
      "\u001b[36mINFO\u001b[0m Deleting component 'mymab' at path '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/mymab.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Successfully deleted component 'mymab'       \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks delete default -c mymab && \\\n",
    "    ks component rm mymab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Model with Outlier Detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Writing component at '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/myout.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Applying seldondeployments seldon.myout      \n",
      "\u001b[36mINFO\u001b[0m Creating non-existent seldondeployments seldon.myout \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks generate seldon-outlier-detector-v1alpha2 myout --image seldonio/mock_classifier:1.0 && \\\n",
    "    ks apply default -c myout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---\r\n",
      "apiVersion: machinelearning.seldon.io/v1alpha2\r\n",
      "kind: SeldonDeployment\r\n",
      "metadata:\r\n",
      "  labels:\r\n",
      "    app: seldon\r\n",
      "    ksonnet.io/component: myout\r\n",
      "  name: myout\r\n",
      "  namespace: seldon\r\n",
      "spec:\r\n",
      "  annotations:\r\n",
      "    deployment_version: v1\r\n",
      "    project_name: myout\r\n",
      "  name: myout\r\n",
      "  predictors:\r\n",
      "  - annotations:\r\n",
      "      predictor_version: v1\r\n",
      "    componentSpecs:\r\n",
      "    - spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/mock_classifier:1.0\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: myout\r\n",
      "          volumeMounts: []\r\n",
      "        imagePullSecrets: []\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "        volumes: []\r\n",
      "    - spec:\r\n",
      "        containers:\r\n",
      "        - image: seldonio/outlier_mahalanobis:0.3\r\n",
      "          imagePullPolicy: IfNotPresent\r\n",
      "          name: outlier-detector\r\n",
      "        terminationGracePeriodSeconds: 1\r\n",
      "    graph:\r\n",
      "      children:\r\n",
      "      - children: []\r\n",
      "        endpoint:\r\n",
      "          type: REST\r\n",
      "        name: myout\r\n",
      "        type: MODEL\r\n",
      "      endpoint:\r\n",
      "        type: REST\r\n",
      "      name: outlier-detector\r\n",
      "      type: TRANSFORMER\r\n",
      "    name: myout\r\n",
      "    replicas: 1\r\n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks show default -c myout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"myout-myout-svc-orch\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"myout-myout-svc-orch\" successfully rolled out\n",
      "Waiting for deployment \"myout-myout-outlier-detector-1\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"myout-myout-outlier-detector-1\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/myout-myout-svc-orch\n",
    "!kubectl rollout status deploy/myout-myout-outlier-detector-1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[replicasAvailable:1 name:myout-myout-svc-orch replicas:1] map[name:myout-myout-myout-0 replicas:1 replicasAvailable:1] map[name:myout-myout-outlier-detector-1 replicas:1 replicasAvailable:1]] state:Available]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments myout -o jsonpath='{.status}' -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_AMBASSADOR=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"2f1ji1fq1jrk7i94eje2vrpii7\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": [0.0]\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07750755488143103]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "r = rest_request_ambassador(\"myout\",API_AMBASSADOR)\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"fk8648qmqh85h46i1lq3bfch5n\"\n",
       "  tags {\n",
       "    key: \"outlierScore\"\n",
       "    value {\n",
       "      list_value {\n",
       "        values {\n",
       "          number_value: 0.0\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  routing {\n",
       "    key: \"outlier-detector\"\n",
       "    value: -1\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.06528283140780505\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"myout\",API_AMBASSADOR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36mINFO\u001b[0m Deleting seldondeployments seldon.myout      \n",
      "\u001b[36mINFO\u001b[0m removing environment component                \u001b[36mcomponent-name\u001b[0m=myout\n",
      "\u001b[36mINFO\u001b[0m Removing component parameter references ...  \n",
      "\u001b[36mINFO\u001b[0m Deleting component 'myout' at path '/home/clive/work/seldon-core/fork-seldon-core/notebooks/my-ml-deployment/components/myout.jsonnet' \n",
      "\u001b[36mINFO\u001b[0m Successfully deleted component 'myout'       \n"
     ]
    }
   ],
   "source": [
    "!cd my-ml-deployment && \\\n",
    "    ks delete default -c myout && \\\n",
    "    ks component rm myout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tear down"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd my-ml-deployment && ks delete default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm -rf my-ml-deployment"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
